Programmed audio-visual presentation of information for instruction in the operation of keyboard controlled instruments

ABSTRACT

A system for instruction in the operation of a keyboard controlled instrument includes means for playback of synchronized prerecorded instructional programs including audio instructions and pictorial representations of the proper position of an operator&#39;&#39;s fingers when carrying out the audio instructions. In alternative embodiments, the pictorial representation is presented either proximate to the respective keys or is projected directly upon the key surface, but in either instance the position of the pictorially represented keys corresponds to that of the actual keys of the keyboard. The effect of the pictorial representation may be reinforced by additional indicators in the form of lights associated with the individual keys and projected thereon.

United States Patent [191 Schmoyer [4 1 Apr. 3, 1973 [54]PROGRAMMED-AUDIO-VISUAL 2,917,834 12 1959 PRESENTATION OF INFORMATION3.32323 12/132;

3:6l9:886 11/1971 Saave et a1. ..29 203 CONTROLLED INSTRUMENTS [76]Inventor: Arthur R. Schmoyer, 11708 River Road, Potomac, Md. 20854 [22]Filed: June 10, 1971 [21] Appl. No.: 151,729

[51] Int. Cl. ..G09b 15/04 [58] Field of Search ..35/5, 6, 9 R; 84/470,477, 84/478, 485

[56] References Cited UNITED STATES PATENTS 3,377,716 4/1968 Schmoyer"35/6 UX 3,610,086 10/1971 Decker ..84/470 3,353,435 11/1967 Schmoyer.....84/478 3,482,480 12/1969 Decker ..84/470 X 3,577,824 5/1971 Lavan..84/470 3,153,365 10/1964 RedmonL'... ..84/478 3,403,591 10/1968Weitzner ..84/485 Primary Examiner-Robert W. Michell AssistantExaminer-J. H. Wolff Attorney-Smith, Michael, Bradford & Gardiner ['57]ABSTRACT A system for instruction in the operation of a keyboardcontrolled instrument includes means for playback of synchronizedprerecorded instructional programs including audio instructions andpictorial representations of the proper position of an operators fingerswhen carrying out the audio instructions. In alternative embodiments,the pictorial representation is presented either proximate to therespective keys or is projected directly upon the key surface, but ineither instance the position of the pictorially represented keyscorresponds to that of the actual keys of the keyboard. The effect ofthe pictorial representation may be reinforced by'additional indicatorsin the form of lights associated with the individual keys and projectedthereon.

12 Claims, 7 Drawing Figures PATENTEDAPR 3 I975 SHEET 1 OF 3 INVENTOIArthur R. Sc/rmoyer Fig.

ATTORNEYS PATENTEDAPR3 ms 3.724.097

sum 2 OF 3 Fig. 2

J INVE N 70/? Anhur R. Schmoyer M M Mnmnnns PROGRAMMED AUDIO-VISUALPRESENTATION OF INFORMATION FOR INSTRUCTION IN THE OPERATION OF KEYBOARDCONTROLLED INSTRUMENTS ENVIRONMENT OF INVENTION device for teaching theoperation of a keyboard controlled instrument, such as a typewriter,keypunch machine, calculator, piano, or organ, wherein the apparatus isactivated by a pre-recorded magnetic tape bearing a program includingboth audio instructions and means for synchronizing visual instructionstherewith.

BACKGROUND OF INVENTION PRIOR ART Apparatus of the aforementioned typehas been suggested wherein a program of audio instructions and codedelectrical impulse producing signals are used to control reproducingmechanism to present synchronized audible and visual stimuli to thestudent. One such apparatus is disclosed in my US. Pat. No. 3,377,7l6,issued Apr. 16, 1968, and another is the subject of application Ser. No.80,354, filed Oct. 13, 1970 (now U.S. Pat. No. 3,664,036) by John E. E.Boswell, Jr., and Jesse H. Miner.

The visual portion of the instructional program in each of theaforementioned disclosures is presented in the form of lights or otherindicators which may be proximate to or built into the actual keys ofthe OBJECTS OF INVENTION The apparatus of the aforedescribed prior artsuffers from either of two deficiencies. First, the visual informationis presented as indications indicativeonly of the position of the key tobe depressed and is not helpful to the student in determining theposition of his-hand and fingers, or if indicative of the identity ofthe fingerin which action is solicited, such indication is provided bynumber, color, etc., and must be translated" in the students mind, thuscomplicating and slowing the thought process. Secondly, where pictorialrepresentations have been presented, they are so remote from thekeyboard on which the students response is solicited that the studentmust mentally transport the information from the display to the blankkeyboard. Moreover, the lack of physical correspondence between thepictorial represented keys and the keyboard itself requires that thestudent orient his mental picture of the hand position along thekeyboard in order to associate the fingers with the appropriateindividual keys.

In contradistinction to the above, it is among the objects of thisinvention to provide a method and apparatus for programmed audio-visual.presentation of instructional information in the operation of keyboardcontrolled instruments in which:

l. a student receives audible instructions and synchronized visualrepresentations of an operators hand in the position which he is beinginstructed to assume,

2. the pictorial representation is presented closely adjacent to ordirectly on the surface of the keys, 3. the position of the keys in thepictorial representation correspond to that of the keys of the keyboarditself,

4. the student may determine the accuracy of his performance in allaspects of form by directly comparing the position and form of his handwith that of the pictorial representation, and

5. the effect of the pictorial representation is reinforced byadditional visual stimuli in the form of light indicators.

DESCRIPTION OF DRAWINGS These and other self-evident advantages of thisinvention will be better understood by a consideration of the ensuingspecification and accompanying drawings, in which FIG. 1 is aperspective view of the apparatus of this invention,

FIG. 2 is a front elevational view of a keyboard illustrating a studentshand approaching the keyboard in correspondence to the pictorialrepresentation therebehind, 1

FIG. 3 is a plan view of the keyboard of this invention,

FIG. 4 is a cross section of the light bar taken on section IV-'IV ofFIG. 6,

FIG. 5 is a cross section of the light bar taken on section V-V of FIG.6,

FIG. 6 is a perspective view of the light bar,

FIG. 7 is a diagrammatic representation of a second embodiment of theinvention wherein the pictorial representation is presented directly onthe key surfaces.

DESCRIPTION OF APPARATUS OF INVENTION Referring to FIG. 1, a preferredembodiment of the system .of this invention comprises a student'sconsole generally indicated at 1, a tape player 2, a' decoder 3,

and an audio reproducing means 4. v. The tape player 2 is of anyconventional design, and is depicted diagrammatically inasmuch as thestructure thereof forms no part of this invention. In practice, thecassette type of tape player has been found preferable. It functions toread out at least a portion of a program of synchronized audio andvisual instructional information pre-recorded on tape in a form whichincludes the basic audio instructional information and further includes,in any one of three different embodiments, either I. cueing signals foroperation of means, such as a film strip projector, for reproducing thevisual information portion of the program slaved to synchronism with theaudio, 2. cueing signals as in (l) and additional coded signals foroperation of light displays in the manner .Of the aforementionedapplication Ser. No. 80,354 (now US. Pat. No. 3,664,036) and in US. Pat.No. 3,377,7 l 6, or

3. a complete program of audio and visual information recorded'in theform used in cartridge television recording.

The invention will befurther described as it is practiced in the secondembodiment referred to above, it being understood that in the firstembodiment the light bar and associated decoding equipment is merelyeliminated from the system, and in the third embodiment, the visualreproduction is directly from the tape, wherein a television reproducingmeans including a cathode ray tube (not shown) replaces the projectorand screen illustrated in the drawings.

In the second embodiment listed above, signals from the tape are readout at the tape deck and fed through cable to the decoder 3, also of thetype generally set forth in the aforementioned application and patent,This decoder serves to distinguish the audio portion In the operation ofthis system, yet to be described, it often becomes desirable toreinforce the pictorial representation with light indicators designatingthe locations of keys directly on the keyboard itself. To this end, thesystem of this invention includes a light bar'20 positioned on thekeyboard as illustrated in FIG. 1 and shown in more detail in FIGS. 4, 5and 6. The light bar 20 comprises a block 21 comprising an upper surface22 on which appears indicia appropriate to the identity of the keys overwhich the light bar is to be placed. The bottom surface of the block 21is configured to key the light bar to proper placement on the keys. Tothis end, those portions of the bar overlying the white keys have abottom surface 23 lying in a first plane while those portions overlyingthe black keys have a bottom portion 24 in a plane elevated from thefirst plane to acfrom the visual cueing and informational portion of theprogram, and passes the audio signals through cable 6 to audioreproducing means 4 comprising a conventional audio amplifier (notshown) and loudspeaker 7.

Within the decoder 3 cueing signals are detected and are passed throughcable 8 to the film strip advance mechanism of a conventional projector9. The projector is arranged to project pictorial representations of thevisual instructional portion of the program on a screen 11. Theplacement of the screen 11 is an essential feature of the invention, asits location at a-point contiguous to the keyboard itself permitspresentation of images of individual keys in a position of precisecorrespondence with the respective keys of the keyboard itself.Referring to FIG. 2, the screen is placed in juxtaposed relationship tothe keyboard, i.e., directly above and to the rear of the keyboarditself, the lower edge of the screen being substantially contiguous tothe keyboard itself. The pictorial representation projected on thescreen is carefully dimensioned and framed so as to orient the image ofany given key in direct correspondence with the position of therespective key of the keyboard. Note, for instance, that the key .6,designated by reference numeral 12, is directly under the imagerepresentative of key G, the image being designated by reference numeral13.

In an alternative embodiment illustrated in FIG. 7, the projector 9 isso oriented as to project the image directly upon the surface of thekeys themselves, as through an optical path including a reflector 14. Inthis case, the surface of the keys takes the place of the separatescreen 11 and functions'as what is referred to in the ensuing claims asmeans substantially contiguous to the keyboard for displayingpictorially represented keys, it being understood that the termsubstantially contiguous is considered generic to either a screensituated so closely proximate to the keyboard that one edge of one isvirtually in contact with an edge of the other, or to a structurewherein the key surface itself functions as the screen and wherein thekey images are actually congruent to the individual keys and thus wheremore than one edge is contiguous thereto. Similarly, the term positionalcorrespondence is generic to a representation wherein the position ofthe sides of each key image is either in vertical alignment or iscongruent to the sides of the corresponding'key-on the keyboard.

commodate the elevated touch surface of black keys on a normal keyboard.With this bottom configuration, the light bar will fit in only oneposition within the span'of an octave on the keyboard and, once inplace, will be retained in properly indexed position without need of anyfasteners or other retaining means.

The light bar further includes front surfaces 25, 26, etc., overlyingeach individual key, the surfaces 26 overlying black keys beingpreferably set back from the surfaces 25 which overlie white keys. Eachfront surface which overlies a significant key, i.e., a key whichfigures in the instructional program, has disposed thereon a light 27set back slightly from the surface 25 or 26 so as to be visible to thestudent only indirectly and so as to project a light beam on the surfaceof the corresponding individual key. By appropriate choice of the lamp,use of light piping (not shown) and dimensions of the setback from thefront surface the projected light may be confined or at leastconcentrated within the bounds of the surface of one key. Note also thatthe setback of the front surfaces 26 overlying black keys provides sidewalls 28 which assist in this regard.

The wiring of the light bar 20 does not comprise a portion of theinvention claimed herein, and hence is not shown, reference again beingmade to US. Pat. No. 3,377,716 for a disclosure thereof. Insofar as thisinvention is concerned, it will suffice to state that coded informationread from the prerecorded tape by player 2 and recognized and decoded at3 is fed to the light bar 20 through cable 29 to energize appropriatelights in synchronized relationship to the other portions of theprogram, namely the audible instructions heard from speaker 7 and thepictorial representations presented at the keyboard by projector 9.Inasmuch as the use of the light bar is optional, it is preferable toconnect it to the system by a readily accessible multi-contact plug 30so as to facilitate its removal when not in use.

DESCRIPTION OF OPERATION OF INVENTION The unit is self-sufficient andrequires no auxiliary equipment, although printed instructions or worksheets prepared in concert with the audio-visual program are helpful. Inuse, the student is situated before the console 1 within audible rangeof the speaker 7. Operation of the apparatus is initiated simply byselecting the proper lesson on tape, and the corresponding film strip,loading them into the tape player and projector, and depressing theplayback control on the tape player.

The pre-recorded program comprises an audio portion recorded on the tapein conventional fashion, and a visual informational portion disposed inpart on the tape and in part on the film strip. The portion disposedupon the film strip comprises the pictorial representation ofinstructional information while that disposed on the tape comprises cuesfor advancing the film strip and coded signals for control of the lightbar.

Considering a logical make-up of an elementary lesson, the tape wouldinitially include an audible description of the keyboard itself, atwhich time a pictorial representation of the bare keyboard is presentedin the manner aforedescribed, i.e., with individual key images in directpositional correspondence to the'position of individual keys of thekeyboard itself. As the audible instruction proceeds to a point wherecertain keys are identified, a cue on the tape actuates the filmadvancing means of the film strip projector in will-known fashion tobring before the projector a similar pictorial representation havinginstructional stimuli in the form of identifying indicia superposed onthe one or more key images being referred to in thenarrative. Thus, thestudents comprehension of the position of the keys is enhanced. Whilethis pictorial representation remains in place, the narrative may referto specific ones of the keys identified therein. At each point in thenarrative where this occurs, a coding signal or signals distinctive ofthat key or keys is present on the tape. Each coding signal or signalswould, of course, have appropriate switching signals associatedtherewith when the apparatus and method of the aforementionedapplication Ser. No. 80,354 is practiced. This signal, acting throughdecoder 3, is effective to activate the light 27 on light bar whichoverlies the said one key or keys being referred to, thus reinforcingthe visible instructional stimulus provided by the pictorialrepresentation.

As the instruction proceeds, there comes a time when the studentreceives instructions to place his fingers over specific keys. Here, theform of the students hand becomes important. Hence, a cue causes furtheradvancement of the film strip to a frame which includes an image of anoperators hand superposed on the keyboard image, as illustrated in FIG.2. The student may then compare the manner-in which he is placing hishand on the keyboard with the pictorial representation appearing eitherjust adjacent thereto (FIG. 2) or congruent thereto (FIG. 7), andcorrect the form or position of his'hand to bring it into exactcompliance with the pictorial representation. Again, the visual stimulusmay be further enhanced by the appropriate activation of the light bar.

In the foregoing description, reference has been made to the use of filmstrip in which advancement occurs intermittently in response toindividual cues on the tape, It should, however, be understood thatequivalent methods of presentation such as a motion picture projectorslaved to synchronism with the audio or carrying the audio and video onthe same media, or cartridge television carrying the audio and video onthe same media are considered full equivalent means. The specific meansdescribed throughout the ensuing specification are set forth as merelyexemplary and not as limits on the scope of my invention, the scope ofwhich is to be determined by a consideration of the following claims:

Iclaim:

l. A system for instruction in the operation of a keyboard controlledinstrument, said system comprism a. at least one keyboard includingkeys,

b. means including a pre-recorded program of synchronized audio andvisual instructional information,

c. means for audibly reproducing the audio instructional content of saidprogram, means for deriving from at least a portion of the visualinformational content of said program a changeable sequence of pictorialrepresentations of instructional stimuli superposed on key images in aposition indicative of the response being solicited of a student, saidkey images comprising a reproduced likeness of key surfaces,

e. means substantially contiguous to said keyboard for displaying saidpictorial representations, and

f. means projecting said representations on said display means with saidkey images in direct positional correspondence with the keys of saidkeyboard.

2. A system for instruction in the operation of a keyboard controlledinstrument as set forth in claim 1, wherein said display means isdisposed in juxtaposition to said keyboard and said direct positionalcorrespondence comprises a vertical alignment between said pictoriallyrepresented key images and said keys.

3. A system for instruction in the operation of a keyboard controlledinstrument as set forth in claim 1 wherein said means for displayingcomprises the touch surface of the keys and said direct positionalcorrespondence is a congruence between said pictorially key images andsaid keys.

4. A system for instruction in the operation of a keyboard controlledinstrument as set forth in claim 1 wherein said prerecorded programcomprises a magnetic tape including said audio information and controlsignal components of said visual information, and further comprises afilm strip including pictorial components of said visual instructionalinformation.

5. A system for instruction in the operation of a keyboard controlledinstrument as set forth in claim 1 wherein said instructional stimuliinclude an image of an operator's hand in direct positionalcorrespondence to the hand position being solicited as said response ofsaid student.

6. A system for instruction in the operation of a keyboard controlledinstrument as set forth in claim 5 wherein said instrument is an organ.

7. A system for instruction in the operation of a keyboard controlledinstrument as set forth in claim 1 and additionally including aplurality of light indicators, each said indicator overlying anindividual key of said keyboard, the instructional information contentof said program including control signals in addition to said pictorialinformation, and means responsive to said control signals for actuatingsaid light indicators.

8. A method of providing instruction in the operation of a keyboardcontrolled instrument, said method comprising the steps of A. providinga pre-recorded program of audio and visual instructional information,

B. audibly reproducing the audio content of said program,

tent of said program a pictorial representation of instructional stimulisuperposedron images com prising a reproduced likeness of a keyboard,

D. projecting said pictorial representation and displaying same in aposition substantially contiguous to the keyboard with said key imagesin direct positional correspondence with the keys of the keyboard, and

E. advancing said program while maintaining said audio and visualcontent in synchronism..

9. The method of providing instruction in the operation of a keyboardcontrolled instrument as set forth in claim 8 wherein step C comprisesderiving said instructional stimuli in the form of an operators handsuperposed on said key images in direct position correspondence to thehand position being solicited as a response of a student.

-10. The method of providing instruction in the operation of a keyboardcontrolled instrument as set forth in claim 8 wherein step D comprisesprojecting said pictorial representation on the touch surface of saidkeyboard so that said direct positional correspondence manifests itselfas a congruent relationship between individual key images and thecorresponding keys of the keyboard.

11. The method of providing instruction in the operation of a keyboardcontrolled instrument as set forth in claim 8 wherein step comprisespresenting" said pictorial representation in juxtaposition to saidkeyboard so that said direct positional correspondence manifests itselfas a vertical alignment between said key images and said keys.

12. The method of providing instruction in the operation of a keyboardcontrolled instrument as set forth in claim 8 wherein said programcomprises instructional information for playing an organ.

1. A system for instruction in the operation of a keyboard controlledinstrument, said system comprising: a. at least one keyboard includingkeys, b. means including a pre-recorded program of synchronized audioand visual instructional information, c. means for audibly reproducingthe audio instructional content of said program, d. means for derivingfrom at least a portion of the visual informational content of saidprogram a changeable sequence of pictorial representations ofinstructional stimuli superposed on key images in a position indicativeof the response being solicited of a student, said key images comprisinga reproduced likeness of key surfaces, e. means substantially contiguousto said keyboard for displaying said pictorial representations, and f.means projecting said representations on said display means with saidkey images in direct positional correspondence with the keys of saidkeyboard.
 2. A system for instruction in the operation of a keyboardcontrolled instrument as set forth in claim 1, wherein said displaymeans is disposed in juxtaposition to said keyboard and said directpositional correspondence comprises a vertical alignment between saidpictorially represented key images and said keys.
 3. A system forinstruction in the operation of a keyboard controlled instrument as setforth in claim 1 wherein said means for displaying comprises the touchsurface of the keys and said direct positional correspondence is acongruence between said pictorially key images and said keys.
 4. Asystem for instruction in the operation of a keyboard controlledinstrument as set forth in claim 1 wherein said prerecorded programcomprises a magnetic tape including said audio information and controlsignal components of said visual information, and further comprises afilm strip including pictorial components of said visual instructionalinformation.
 5. A system for instruction in the operation of a keyboardcontrolled instrument as set forth in claim 1 wherein said instructionalstimuli include an image of an operator''s hand in direct positionalcorrespondence to the hand position being solicited as said response ofsaid student.
 6. A system for instruction in the operation of a keyboardcontrolled instrument as set forth in claim 5 wherein said instrument isan organ.
 7. A system for instruction in the operation of a keyboardcontrolled instrument as set forth in claim 1 and additionally includinga plurality of light indicators, each said indicator overlying anindividual key of said keyboard, the instructional information contentof said program including control signals in addition to said pictorialinformation, and means responsive to said control signals for actuatingsaid light indicators.
 8. A method of providing instruction in theoperation of a keyboard controlled instrument, said method comprisingthe steps of A. providing a pre-recorded program of audio and visualinstructional information, B. audibly reproducing the audio content ofsaid program, C. deriving from at least a portion of the visual contentof said program a pictorial representation of instructional stimulisuperposed on images comprising a reproduced likeness of a keyboard, D.projecting said pictorial representation and displaying same in aposition substantially contiguous to the keyboard with said key imagesin direct positional correspondence with the keys of the keyboard, andE. advancing said program while maintaining said audio and visualcontent in synchronism.
 9. The method of providing instruction in theoperation of a keyboard controlled instrument as set forth in claim 8wherein step C comprises deriving said instructional stimuli in the formof an operator''s hand superposed on said key images in direct positioncorrespondence to the hand position being solicited as a response of astudent.
 10. The method of providing instruction in the operation of akeyboard controlled instrument as set forth in claim 8 wherein step Dcomprises projecting said pictorial representation on the touch surfaceof said keyboard so that said direct positional correspondence manifestsitself as a congruent relationship between individual key images and thecorresponding keys of the keyboard.
 11. The method of providinginstruction in the operation of a keyboard controlled instrument as setforth in claim 8 wherein step D comprises presenting said pictorialrepresentation in juxtaposition to said keyboard so that said directpositional correspondence manifests itself as a vertical alignmentbetween said key images and said keys.
 12. The method of providinginstruction in the operation of a keyboard controlled instrument as setforth in claim 8 wherein said program comprises instructionalinformation for playing an organ.